1. Field of the Invention
The present invention relates to an image processing apparatus, an imaging apparatus, and an image processing method, and a computer program. Specifically, the present invention relates to an image processing apparatus, an imaging apparatus, an image processing method, and a computer program in which a high-quality output image is generated on the basis of two types of images, that is, a long-time exposure image and a short-time exposure image.
2. Description of the Related Art
Solid-state imaging elements such as a charge-coupled device (CCD) image sensor and a complementary metal oxide semiconductor (CMOS) image sensor used in video cameras, digital still cameras, and the like accumulate electric charge corresponding to an amount of light incident thereon, and output an electrical signal corresponding to the accumulated electric charge, in other words, performs photoelectric conversion. However, there is a limit to the amount of electric charge that can be accumulated in photoelectric conversion elements. When an amount of light more than a certain level is received, the amount of accumulated electric charge reaches a saturation level so that a subject area brighter than a certain value is set to a level of saturated brightness. That is, a “white-out” problem occurs.
To avoid such a white-out phenomenon, a period during which electric charge is accumulated in a photoelectric conversion element is controlled in accordance with a change in ambient light or the like to adjust exposure time to optimally control sensitivity. For example, when a bright subject is photographed, a shutter is released at a high speed to reduce the exposure time to reduce the period during which electric charge is accumulated in the photoelectric conversion element so that an electrical signal is output before the amount of accumulated electric charge has reached the saturation level. This allows output of images with accurate reproduction of grayscale in accordance with subjects.
In the photography of a subject including both bright and dark areas, however, high-speed shutter release does not ensure sufficient exposure time in the dark areas, resulting in deterioration of signal-to-noise (S/N) ratio and low image quality. In a photographed image of such a subject including both bright and dark areas, brightness levels of the bright and dark areas are accurately reproduced by increasing the exposure time in pixels with a small amount of light incident on the image sensor to realize high S/N ratio while avoiding saturation in pixels with a large amount of incident light.
A method for achieving such accurate reproduction is described in, for example, “A Linear-Logarithmic CMOS Sensor with Offset Calibration Using an Injected Charge Signal,” IEEE International Solid-State Circuits Conference (ISSCC) 2005, pp. 354, February 2005. Specifically, as shown in FIG. 1, an amplification-type image sensor includes pixels 100 arranged in a matrix, each pixel having a photodiode 101, a transfer transistor 102, a reset transistor 103, an amplification transistor 104, and a selection transistor 105. To turn off the transfer transistor 102, a voltage to be applied to a control electrode of the transfer transistor 102 is set to a level Vtrg that allows excess electrons over a certain value to flow to a floating diffusion (FD) node 106, ether than a standard level for completely turning of the transfer transistor 102.
When the number of electrons accumulated in the photodiode 101 exceeds the level Vtrg, leakage of the excess electrons to the FD node 106 starts to occur in a subthreshold region. The leakage operates within the subthreshold region, and the number of electrons that remains in the photodiode 101 is a logarithmic response.
As shown in FIG. 2, after a reset operation in a period T0, accumulation of electrons is executed while the voltage Vtrg is still applied to the control electrode of the transfer transistor 102. In a period T1 in which the number of accumulated electrons is small, all the electrons are stored in the photodiode 101. When the number of accumulated electrons exceeds the level Vtrg, the electrons start to leak to the FD node 106 as indicated by a period T2.
Due to the leakage in the subthreshold region, electrons are accumulated with a logarithmic characteristic with respect to the incident light intensity even when the accumulation continues (in a period T3). In a period T4, the electrons overflowed to the FD node 106 are reset, and all electrons stored in the photodiode 101 are read by a complete transfer. FIG. 3 shows a relationship between the incident light intensity and the number of output electrons. With respect to an incident light intensity exceeding an upper limit Qlinear of a linear region defined by the voltage Vtrg, the number of output electrons is determined by a logarithmic response.
Although the achievement of a dynamic range of 124 dB is reported in the aforementioned related art, the saturation level of the linear region in which a high S/N ratio can be achieved is less than or equal to a half of a standard saturation level Qs. Further, while a significantly wide dynamic range is achieved by a logarithmic response, a logarithmic response circuit is susceptible to threshold variations or the like of the transfer transistor 102. Therefore, fixed pattern noise as large as 5 mV for the logarithmic region, compared with a fixed pattern noise of 0.8 mV for the linear region, remains in the wide dynamic range even if the threshold variations are canceled.